Recently, digital broadcasting in which image data and so on are transmitted and received as digital data is drawing the attention. An advantage of the digital broadcasting is that more program data (hereinafter referred to as programs) can be transmitted over the same transmission channel as compared with analog broadcasting. This is due in large part to the fact that image data can be compressed before being transmitted. As the method for compressing image data, there is, for example, a bidirectional predictive encoding system adopted in MPEG (Moving Picture Experts Group) standards. In this bidirectional predictive encoding system, three types of encoding, i.e., intra-frame encoding, inter-frame forward predictive encoding, and bidirectional predictive encoding are performed. Images obtained by using respective encoding types are called I picture (intra encoded picture), P picture (predictive encoded picture), and B picture (bidirectionally predictive encoded picture), respectively.
FIG. 1 is a block diagram showing an example of a configuration of a digital broadcasting system using an image encoding device of the bidirectional predictive encoding system according to the MPEG standards. A digital broadcasting system 300 includes a plurality of image encoding devices 3011 through 301n (where n is an integer value of minimum 2) for compressing and encoding program data such as image data, a multiplexer 302 connected to these image encoding devices 3011 through 301n so as to multiplex data compressed and encoded by the respective image encoding devices 3011 through 301n, and a modulator 303 for modulating output image data multiplexed by the multiplexer 302. In the digital broadcasting system 300, image data are compressed and encoded by n image encoding devices 3011 through 301n, and outputted to the multiplexer 302. The multiplexer 302 multiplexes compressed and encoded data inputted from the image encoding devices 3011 through 301n, and outputs the multiplexed data to the modulator 303 with a fixed data rate (speed) of, for example, approximately 30 Mbps. The compressed and encoded data supplied from the multiplexer 302 is modulated by the modulator 303, and transmitted to each home 305 via a broadcasting satellite 304.
FIG. 2 is a block diagram showing an example of a configuration of the image encoding device illustrated in FIG. 1. The image encoding device 301 (which is representative of 3011 through 301n) includes: a subtraction circuit 310 for deriving a difference between inputted image data S101 and predicted image data; a DCT circuit 311 for conducting DCT on output data of the subtraction circuit 310 by each DCT block and outputting a DCT coefficient; a quantization circuit 312 for quantizing output data of the DCT circuit 311; a variable length encoding circuit 313 for conducting variable length encoding on output data of the quantization circuit 312; a buffer memory 314 for temporarily holding output data of the variable length encoding circuit 313 and outputting it as compressed image data S102 comprising a bit stream having a fixed bit rate; an inverse quantization circuit 315 for conducting inverse quantization on the output data of the quantization circuit 312; an inverse DCT circuit 316 for conducting inverse DCT on output data of the inverse quantization circuit 315; an addition circuit 317 for adding output data of the inverse DCT circuit 316 to the predicted image data and outputting a resulting sum; a motion compensation circuit 318 for holding the output data of the addition circuit 317, conducting motion compensation on the output data on the basis of a motion vector, and outputting predicted image data to the subtraction circuit 310 and the addition circuit 317; a bit rate control unit 319 for controlling a target code quantity on the basis of generated bit quantity data S103 supplied from the variable length encoding circuit 313 so as to make the compressed image data S102 outputted from the buffer memory 314 become a bit stream having a fixed rate; and a motion detection circuit 309 for detecting the motion vector on the basis of the inputted image data S101 and transmitting the detected motion vector to the motion compensation circuit 318.
In the image encoding device 301 shown in FIG. 2, the image data S101 is inputted to the subtraction circuit 310. The output signal of the subtraction circuit 310 is inputted to the DCT circuit 311, and subjected to DCT. The output signal of the DCT circuit 311 is quantized by the quantization circuit 312, and subjected to variable length encoding in the variable length encoding circuit 313. The output data of the variable length encoding circuit 313 is temporarily held by the buffer memory 314 and outputted as the compressed image data S102.
Furthermore, from the variable length encoding circuit 313, the generated bit quantity data S103 is outputted to the bit rate control unit 319. On the basis of the generated bit quantity data S103, the bit rate control unit 319 determines the target code quantity. According to the target code quantity, the bit rate control unit 319 controls quantization characteristic in the quantization circuit 312.
By the way, in image compression and encoding in digital broadcasting, it is necessary to keep the image quality high while suppressing the data quantity being compressed and encoded to be the transmission capacity of a transmission channel or less.
There is a technique called “statistical multiplexing”, as a method for letting more programs flow through a transmission channel having a predetermined transmission capacity. The statistical multiplexing is a technique for transmitting a larger number of programs by dynamically changing transmission rates of respective programs. According to this statistical multiplexing, for example, the transmission rates are lowered on programs which transmission rates reduction do not cause a conspicuous image quality degradation, so as to make it possible to transmit a larger number of programs. The statistical multiplexing utilizes the fact that such portions (time periods) of respective programs in which image quality degradation are conspicuous, rarely overlap with one another simultaneously. Therefore, when such a portion of a certain program which makes conspicuous image quality degradation is transmitted, image quality degradation is not conspicuous in other programs in many cases even if the code rate is lowered. Accordingly, it is possible to lower code rates of other programs and assign the higher code rate to such a program that image quality degradation is conspicuous. By using the statistical multiplexing, a larger number of programs than the ordinary case can thus be transmitted.
In such a statistical multiplexing, a bit rate assigning technique for determining a bit rate quantity as code rates to be assigned to respective programs is an important factor in determining the image quality and so on. According to a representative technique proposed heretofore as a bit rate assigning technique, quantization steps used in respective programs are monitored and they are subjected to feedback control so that the quantization steps will become the same in all programs or quantization steps will have weights preset for respective programs. In such a feedback control, after an image is encoded, the next bit rate is determined on the basis of a quantization step used at the time of encoding the image. Therefore, when the pattern is abruptly changed to a difficult pattern caused by such as scene change, a countermeasure will be delayed, resulting in a problem that image distortion occurs significantly.
On the other hand, in order to solve the problem of such a countermeasure delay of the control system due to feedback control, the present applicant has proposed a technique called feedforward control, in which, encoding difficulty representing the difficulty of the encoding regarding an image to be encoded, is derived beforehand, and bit rates of respective programs are determined according to this encoding difficulty. This feedforward control technique is implemented basically by distributing a total bit rate after being multiplexed among respective programs according to ratios of encoding difficulty data read from the respective programs beforehand. Allocation of bit rates to respective programs according to ratios of the encoding difficulty data is determined by proportional allocation as represented by the following expression (1).Ri=(Di/ΣDk)×Total—Rate  (1)
In the expression (1), Ri means a bit rate of a program number i, Di encoding difficulty of the program per unit time, Total—Rate a total bit rate, and Σ a sum total over k=1 to L (where L is the total number of programs).
Alternatively, allocation of bit rates to respective programs is determined as represented by the following expression (2) by adding weight coefficients Wi to respective programs.Ri={Wi×Di/Σ(Wk×Dk)}×Total—Rate  (2)
In this way, the bit rates are proportionally allocated according to the encoding difficulty values.
FIG. 3 is a block diagram showing an example of a configuration of a multiplexing device using the above described statistical multiplexing. This multiplexing device 330 includes a plurality of image encoding devices 3311 to 331n for conducting compression encoding on respective inputted programs P1 to Pn, a statistical multiplexing controller 332 connected to n image encoding devices 3311 to 331n to control the image encoding devices 3311 to 331n, and a multiplexer 333 for multiplexing compressed and encoded data St1 to Stn compressed and encoded by the image encoding devices 3311 to 331n. The image encoding devices 3311 to 331n in advance derive encoding difficulties D1 to Dn each of which represents the difficulty of encoding with regard to an image to be encoded, and output the encoding difficulties D1 to Dn to the statistical multiplexing controller 332. By distributing the total bit rate after being multiplexed according to the ratios of these encoding difficulties D1 to Dn, the statistical multiplexing controller 332 determines target bit rates for the respective programs P1 to Pn according to the ratios of the encoding difficulties D1 to Dn, and outputs control data CR1 to CRn such as the target bit rates to the image encoding devices 3311 to 331n respectively. On the basis of the control data CR1 to CRn such as the target bit rates supplied from the statistical multiplexing controller 332, each of the image encoding devices 3311 to 331n conducts compression encoding on the programs P1 to Pn, and outputs compressed and encoded data St1 to Stn to the multiplexer 333. The multiplexer 333 multiplexes the respective compressed and encoded data St1 to Stn which were inputted, generating image data Sm to be outputted, and outputs the image data Sm to the modulator 303 shown in FIG. 1.
FIG. 4 is a block diagram showing an example of a configuration of the image encoding devices illustrated in FIG. 3. In this image encoding device 331 (which is representative of 3311 to 331n), the same components as those of the image encoding device 301 are denoted by same marks, and description thereof will be omitted. In this image encoding device 331, the statistical multiplexing controller 332 controls the target code quantity, instead of the bit rate control unit 319 of the image encoding device 301 shown in FIG. 2. The motion detection circuit 309 in this image encoding device 331 outputs a ME residual to the statistical multiplexing controller 332 as encoding difficulties D while deriving a motion vector. In brief, the ME residual is the sum total of absolute values or the sum total of squared values of motion prediction errors over a whole picture. On the basis of the encoding difficulties D supplied from the motion detection circuits 309 of each of the image encoding devices 331, the statistical multiplexing controller 332 conducts control using statistical multiplexing, generates control data CR such as target code quantities, and outputs the control data CR to quantization circuits 342. On the basis of the control data CR, the quantization circuit 342 quantizes data outputted from the DCT circuit 311, and outputs the quantized data to a variable length encoding circuit 343. By the way, the statistical multiplexing controller 332 is supplied with the encoding difficulties D from each of the image encoding devices 331, and outputs the control data CR to the quantization circuits 342 each of the image encoding devices 331. In FIG. 4, D is representative of D1 to Dn, and CR is representative of CR1 to CRn.
Each of FIGS. 5A to 5C shows an example of a bit rate change in each of the image encoding devices included in the multiplexing device using the statistical multiplexing. FIG. 5A shows a bit rate change in the image encoding device 3311. FIG. 5B shows a bit rate change in the image encoding device 3312. FIG. 5C shows a bit rate change in the image encoding device 331n. The axis of ordinates represents the bit rate of the image encoding device, and the axis of abscissas represents time. As described above, the statistical multiplexing utilizes the fact that such portions (time periods) of respective programs in which image quality degradation is conspicuous rarely overlap with one another simultaneously. Therefore, when such a portion of a certain program in which image quality degradation is conspicuous is transmitted, image quality degradation is not conspicuous in other programs even if the bit rate is lowered. Accordingly, it is possible to assign some of bit rates of the other programs to such a program that image quality degradation is conspicuous.
As shown in FIGS. 5A to 5C, bit rates assigned to respective image encoding devices 3311 to 331n in which respective programs are inputted are controlled so as to become variable rates in the time axis direction. As shown in FIG. 5A, for example, the bit rate of the image encoding device 3311 is high at, time A. This is because in the image encoding device 3311, the value of the encoding difficulty becomes high at the time A as the image motion is rapid or the pattern is complicated then. Therefore, a high bit rate is assigned to the image encoding device 3311 at the time A. On the other hand, in the image encoding device 331n at time B, the encoding difficulty value becomes low as shown in FIG. 5C, because the image is nearly still or the pattern is simple. Therefore, a low bit rate is assigned to the image encoding device 331n at the time B. Furthermore, the sum total of the bit rates assigned to respective image encoding devices 3311 to 331n must be fixed. For example, at time C of FIGS. 5A to 5C, the sum total of bit rates R1 to Rn respectively assigned to each image encoding devices 3311 to 331n is a fixed quantity, and this becomes the bit rate of the modulator 303 shown in FIG. 1. By using the statistical multiplexing, a larger number of programs than the ordinary case can thus be transmitted.
However, in the multiplexing device 330 shown in FIG. 4, the encoding difficulties D1 to Dn are sent from the respective image encoding devices 3311 to 331n to the statistical multiplexing controller 332. The control data CR1 to CRn such as target bit rates derived from the statistical multiplexing controller 332 on the basis of those encoding difficulties D1 to Dn are sent to the image encoding devices 3311 to 331n, respectively. Therefore, in the statistical multiplexing controller, an input portion and an output portion corresponding to the image encoding devices 3311 to 331n become necessary, resulting in the enlargement of the scale and a problem that data exchange with respective image encoding devices become complicated.
Furthermore, as a multiplexing device using statistical multiplexing, the present applicant previously has proposed a multiplexing device (Japanese Patent Application No. 9-179882), wherein the statistical multiplexing computer conducting bit rate control of feedforward type using a computer for general purpose instead of a statistical multiplexing controller serving as a dedicated device is connected to respective image encoding devices via a network such as Ethernet, and exchange of the encoding difficulties between respective image encoding devices and the statistical multiplexing computer is conducted via the Ethernet. FIG. 6 is a block diagram showing an example of a configuration using the statistical multiplexing computer. In this statistical multiplexing system 400, respective image encoding devices 4021 to 402n output transport streams St1 to Stn, each containing an encoded data sequence corresponding to one channel, to a multiplexer 404. The image encoding devices 4021 to 402n output encoding difficulties D1 to Dn respectively, for controlling compression encoding in the image encoding devices 4021 to 402n, to the statistical multiplexing computer 403. By taking a packet as a unit, the encoding difficulties D1 to Dn are sent from the image encoding devices 4021 to 402n to the statistical multiplexing computer 403 via an Ethernet 405, respectively. Target bit rates Rate1 to Raten respectively assigned to the encoding difficulties D1 to Dn are returned to the respective image encoding devices 4021 to 402n via the same Ethernet 405.
According to the statistical multiplexing system shown in FIG. 6, it thus becomes possible to transmit the encoding difficulties D1 to Dn and the target bit rates Rate1 to Raten efficiently between the image encoding devices 4021 to 402n and the statistical multiplexing computer 403.
However, in the statistical multiplexing system shown in FIG. 6, the network serving as transmission channels between the image encoding devices 4021 to 402n and the statistical multiplexing computer 403, such as the Ethernet 405, goes down in efficiency when the number of transmitted packets increases. Thus there is a possibility that a trouble may occur in the statistical multiplexing system 400 for controlling a large number of image encoding devices 4021 to 402n connected to the network.
Furthermore, the Ethernet 405 is usually used for control systems other than the statistical multiplexing system as well. When a command is sent for the control, there is a possibility that the transmitting of the encoding difficulties D1 to Dn and the target bit rates Rate1 to Raten are affected.